Display control method and display control apparatus

ABSTRACT

Embodiments of the present application disclose a display control method and a display control apparatus, relating to the field of near-to-eye display technologies. The method includes: determining a first display area, in the FOV of a user, of an image displayed on a main display device; and using at least one near-to-eye display device according to the first display area to display augmentation information related to the image. In the method and apparatus of the embodiments of the present application, at least one near-to-eye display device is used to supplement a display that is provided at a suitable viewing distance of a user by a device but cannot fully fill the FOV of the user, and augmentation information is used to fill the FOV of the user as far as possible, so that good immersion media viewing experience can be provided.

RELATED APPLICATIONS

The present application is a continuation-in-part of, and claimspriority to each of, U.S. patent application Ser. No. 15/122,918 (nowU.S. Pat. No. 9,911,214), filed Aug. 31, 2016, and entitled “DISPLAYCONTROL METHOD AND DISPLAY CONTROL APPARATUS,” which is a U.S. NationalStage filing under 35 U.S.C. § 371 of international patent cooperationtreaty (PCT) application No. PCT/CN2014/095323, filed Dec. 29, 2014, andentitled “DISPLAY CONTROL METHOD AND DISPLAY CONTROL APPARATUS”, whichclaims the priority to Chinese Patent Application No. 201410132376.8,filed on Apr. 2, 2014, which applications are hereby incorporated intothe present application herein by reference in their respectiveentireties.

TECHNICAL FIELD

The present application relates to the field of near-to-eye display, andin particular, to a display control method and a display controlapparatus.

BACKGROUND

With the rapid popularization and a continuous increase in the networkconnection speed, mobile computing power, and display quality, portabledevices (mobile phones, tablet computers, and so on) have become a verygood entertainment platform, and consumers spend more and more time onportable devices, for example, playing games and watching videos onportable devices.

Consumers usually prefer portable devices with a relatively small size,but do not like small display screens that can only display smallimages. The field of view (FOV) of a human eye vision system is about180 degree in a horizontal direction, and is about 135 degree in avertical direction. A mobile device can display high-quality mediacontent, but limited by a screen size, the media content may not fillthe whole FOV. For example, according to a display screen size (20 cm×14cm) of a mainstream tablet computer at present, considering that a userviewing distance is 40 cm, in the horizontal direction, only a 30-degreearea of the FOV of the human eye may be filled with image content, whichhas a huge gap to the 180-degree horizontal FOV of the human eye.

SUMMARY

An example, non-limiting objective of the present application is toprovide a display control solution.

To these and/or related ends, in a first example aspect, an embodimentof the present application provides a display control method, including:

determining a first display area, in a FOV of a user, of an imagedisplayed on a main display device; and

using at least one near-to-eye display device according to the firstdisplay area to display augmentation information related to the image.

According to a second example aspect, an embodiment of the presentapplication provides a display control apparatus, including:

a display area determining module, configured to determine a firstdisplay area, in a FOV of a user, of an image displayed on a maindisplay device; and

a display control module, configured to use at least one near-to-eyedisplay device according to the first display area to displayaugmentation information related to the image.

According to a third example aspect, an embodiment of the presentapplication provides a near-to-eye display device, which includes thedisplay control apparatus of the foregoing implementation manner.

In a fourth example aspect of the embodiments of the presentapplication, a computer readable storage device is provided, comprisingat least one executable instruction, which, in response to execution,causes a system comprising a processor to perform operations,comprising:

determining a first display area, in the field of view (FOV) of a user,of an image displayed on a main display device; and

using at least one near-to-eye display device according to the firstdisplay area to display augmentation information related to the image.

In a fifth example aspect of the embodiments of the present application,a device for display control is provided, comprising a processor and amemory, wherein the memory storing a computer executable instruction,the processor being connected with the memory via a communication bus,and when the device is operating, the processor executes or facilitatesexecution of the executable instructions stored by the memory to causethe device to perform operations, comprising:

determining a first display area, in the field of view (FOV) of a user,of an image displayed on a main display device; and

using at least one near-to-eye display device according to the firstdisplay area to display augmentation information related to the image.

In one or more of the embodiments of the present application, at leastone near-to-eye display device is used to supplement a display that isprovided at a suitable viewing distance of a user by a device butdoesn't fully fill the FOV of the user, and augmentation information isused to fill the FOV of the user as far as possible, so as to providegood immersive media viewing experience for the user and to help theuser understanding the target image as well.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an example flowchart of a display control method according toan embodiment of the present application;

FIG. 2 is an example schematic diagram of the FOV of a usercorresponding to a display of a main display device in a display controlmethod according to an embodiment of the present application;

FIG. 3(a) to FIG. 3(c) are example schematic diagrams of thecorresponding FOV of a user when a main display device and a near-to-eyedisplay device perform coordinated display in a display control methodaccording to an embodiment of the present application;

FIG. 4 is an example schematic structural diagram of a display controlapparatus according to an embodiment of the present application;

FIG. 5 is an example schematic structural diagram of a display controlmodule in a display control apparatus according to an embodiment of thepresent application;

FIG. 6 is an example schematic structural diagram of another displaycontrol apparatus according to an embodiment of the present application;

FIG. 7 is an example schematic structural diagram of still anotherdisplay control apparatus according to an embodiment of the presentapplication; and

FIG. 8 is an example schematic structural diagram of a near-to-eyedisplay device according to an embodiment of the present application.

DETAILED DESCRIPTION

Embodiments of the present application are described in further detailbelow with reference to the accompanying drawings (where identical marksin several accompanying drawings represent identical elements) andembodiments. The following embodiments are used to describe the presentapplication, but not used to limit the scope of the present application.

A person skilled in the art can understand that terminologies such as“first” and “second” in the present application are only used todistinguish different steps, devices modules, or the like, which neitherrepresent any specific technical meaning, nor indicate any necessarylogical order between steps, devices, modules, or the like.

The method and apparatus of the embodiments of the present applicationare used to provide immersive media viewing experience for a user whoviews an image (such as a text, a pattern, a video, and a game) with adevice having a display function. This device is referred to as a “maindisplay device” in the embodiments of the present application, which isan main initial device for presenting a target image to a target user,and generally refers to a device, of which a display of a target imageat a suitable viewing distance of a target user doesn't fully fill theFOV of the user. The main display device could be used for presenting areal image or a virtual image of a target image. Such a device is notlimited to a portable device which has a limited display screen size,for example, a mobile phone or a tablet computer, and may also includeany other device which has a relatively limited display capacity ascompared with image quality, for example, a desktop computer or atelevision. The main display device could be a near-to-eye displaydevice as well. A near-to-eye display device is a device that canpresent, at a position near an eye of a user, an image provided by animage source. Such a near-to-eye display device is also referred to as ahead mounted display (HMD), which is, for example, smart glasses, ahelmet, or goggles. Certainly, the near-to-eye display device is notlimited to a head-mounted form, and may also include other possibleportable forms, such as an onboard form and a wearing form. Thenear-to-eye display device can present a virtual image of the image in apenetration manner at a spatial position in the FOV of the user. Theposition of the virtual image presented in a penetration manner iscontrolled, so that final imaging information on the retina of the usernot only includes the image presented by the near-to-eye display device,but also includes an image presented in the rest display area of the FOVof the user, such as a background image or other images displayed by thenear-to-eye display device itself or another display device. Thenear-to-eye display device has a relatively large display area, hasrelatively low costs and power consumption, and is relatively light andsmall. In the method of this embodiment of the present application, theat least one near-to-eye display device may include one or morenear-to-eye display devices, and the near-to-eye display device may be amonocular or binocular near-to-eye display device. In addition, onenear-to-eye display device may have multiple resolution values.

As shown in FIG. 1, a display control method of an embodiment of thepresent application includes:

S110. Determine a first display area, in the FOV of a user, of an imagedisplayed on a main display device.

The FOV of a human eye vision system is about 180 degree in a horizontaldirection, and is about 135 degree in a vertical direction. At a viewingdistance where it is comfortable to view an image, the whole FOV of theuser may be not fully filled by a display of the main display device. Asshown in FIG. 2, the image displayed on the main display device fills afirst display area S1 of the FOV of the user, and there is still a verylarge unfilled region out of the first display area S1 in the FOV of theuser. In the method of this embodiment of the present application, thefirst display area S1 may be determined in many manners. For example,the first display area S1 is determined by detecting relative positionsof the main display device and an eye of the user and with reference tothe size of the main display device; or, an image formed on the fundusof the user is collected, or the image on the main display device iscollected by using a near-to-eye display device, and the first displayarea S1 is then determined by using an image processing and analysismethod.

S120. Use at least one near-to-eye display device according to the firstdisplay area to display augmentation information related to the image,so as to fill the whole FOV of the user as far as possible.

The at least one near-to-eye display device is used to display,according to the first display area, the augmentation informationrelated to the image, so as to supplement the display of the maindisplay device, and enhance viewing experience of the user by fillingthe FOV of the user as far as possible. The augmentation information isan object, a scene, prompt information, or other information that couldhelp the user understanding the target image, and the like that arerelated to the image. By displaying the augmentation information, thenear-to-eye display device can further provide immersive experience forthe user when the user views the image displayed by the main displaydevice.

In conclusion, in the method of this embodiment of the presentapplication, at least one near-to-eye display device is used tosupplement a display that is provided at a suitable viewing distance ofa user by a device but doesn't fully fill the FOV of the user, andaugmentation information can be used to fill the FOV of the user as faras possible, so as to provide good immersive media viewing experiencefor the user and to help the user understanding the target image aswell.

Step S120 may further include:

S121: Determine, according to the first display area, a second displayarea of the at least one near-to-eye display device in the FOV of theuser.

According to different degrees of viewing experience to be provided forthe user, multiple possible second display areas may be determined afterthe first display area is determined. As shown in FIG. 3(a), a seconddisplay area S2 may include: at least a part of the first display areaS1 (as shown in FIG. 3(a), the second display area includes S2 an edgepart of the first display area S1, that is, the second display area S2overlaps an edge of the first display area S1) and a region except thefirst display area S1 in the FOV of the user, which is indicated byoblique lines in FIG. 3(a). Further, as shown in FIG. 3(b), the seconddisplay area S2 may only include the region except the first displayarea S1 in the FOV of the user. It should be noted that vision of ahuman eye has a retina foveal feature: a human eye has different visualsensitivities within the FOV, where a foveal vision region on the retinahas a high visual sensitivity (for example, the foveal vision region issensitive to color, texture, and the like), while a peripheral visionregion is more sensitive to movement information, that is, the fovealvision region is more suitable for viewing the image displayed on themain display device, while the peripheral vision region is moresensitive to the augmentation information. Therefore, the most idealviewing experience can be provided for the user when the second displayarea S2 is the peripheral vision region of the user. Correspondingly, ifthe first display area S1 fills neither more nor less than the wholefoveal vision region, the first display area S1 and the second displayarea S2 are next to each other when the second display area S2 islocated at the peripheral vision region, as shown in FIG. 3(b); if thefirst display area S1 does not fill the whole foveal vision region, agap may exist between the second display area S2 and the first displayarea S1, as shown in FIG. 3(c).

S122. Obtain the augmentation information according to the image.

The augmentation information is an object, a scene, and promptinformation or other information that could help the user understandingthe image, and the like that are related to the image. The augmentationinformation can provide immersive experience for the user when the userviews the image. In the method of this embodiment of the presentapplication, the augmentation information can be obtained in manymanners. For example, the image may include corresponding augmentationinformation, and the augmentation information can be read from the imagedirectly. The augmentation information may be determined according tocontent of the image, for example, the augmentation information isdetermined according to a feature of the image, where the feature of theimage includes at least one piece of the following content: a color,brightness, texture, context, movement, a display attribute (such as aframe frequency), and so on. Determining the augmentation informationaccording to the feature of the image may be a process of determiningthe augmentation information by using an algorithm related to theAugmented Reality Technique (AR) which has been mature in the field. Forexample, a tracking algorithm, an optical flow analysis, brightnesssensing, shape sensing or the like is used to detect a change in a scenein the image, and augmentation information displayed outside a displayscreen of the main display device is determined with reference to anextrapolation algorithm or the like when the scene changes. For anotherexample, context of the image is listened to and comprehended by usingnatural language processing (NLP), and colors and images are used togenerate augmentation information that creates an immersive storyenvironment. The augmentation information in the embodiments of thepresent application is mainly visual information.

S123. Use the at least one near-to-eye display device to display theaugmentation information in the second display area.

After the display area and display content of the at least onenear-to-eye display device are determined, the at least one near-to-eyedisplay device can be used to display the augmentation information inthe second display area, so as to provide immersive media viewingexperience for the user and to help the user understanding the targetimage as well.

It should be noted that, in order to ensure a viewing effect, thespatial position of the augmentation information displayed by the atleast one near-to-eye display device needs to be associated with thespatial position of content displayed by the main display device interms of spatial positioning. When the user rotates or moves the maindisplay device, the first display area changes, and the second displayarea of the at least one near-to-eye display device changes accordingly.

Furthermore, in one possible implementation of the present application,a combination of a real image displayed by the main display device and avirtual image displayed by the at least one near-to-eye display devicemay be presented to the target user finally. For example, the maindisplay device may be a mobile phone, tablet computer, or a desktopcomputer, the at least one near-to-eye display is a smart glasses wornby the user. In another possible implementation of the presentapplication, a combination of a virtual image displayed by the maindisplay device and anther virtual image displayed by the at least onenear-to-eye display device may be presented to the target user finally.In this case, the main display device may be a near-to-eye displaydevice which could or could not be one of the at least one near-to-eyedisplay device. However, in order that a display of the near-to-eyedisplay device does not interfere with a display on the main displaydevice and viewed by the user, but supplements or even improves theviewing experience, the augmentation information displayed by the atleast one near-to-eye display device should be presented as naturally aspossible whether it's a combination of a real image and a virtual imageor of two virtual images presented finally. Therefore, in order toensure the viewing effect, the method of this embodiment of the presentapplication further includes:

S130. Adjust a display parameter of the at least one near-to-eye displaydevice, so as to present the augmentation information to the userclearly and optimize an overall display effect.

The display parameter of the at least one near-to-eye display device maybe adjusted according to a position relationship between the maindisplay device and the eye of the user. For example, the augmentationinformation displayed by the at least one near-to-eye display device ispresented in a same plane as the image displayed by the main displaydevice, and correspondingly, the display parameter of the at least onenear-to-eye display device should be adjusted according to a distancebetween the main display device and the eye of the user.

The display parameter of the at least one near-to-eye display device mayalso be adjusted according to a display feature of the main displaydevice.

The display feature may include: curvature of a display plane of themain display device. For example, when the main display device has acurved display screen, the display parameter of the at least onenear-to-eye display device should be adjusted accordingly, so that thedisplay of the at least one near-to-eye display device and the displayof the main display device can converge in a curved manner naturally andsmoothly.

The display feature may further include: resolution of the main displaydevice. According to the retina foveal feature of the human eye vision,the main display device may display the image with relatively highresolution, while the augmentation information displayed by thenear-to-eye display device does not need resolution as high as that ofthe main display device. Therefore, resolution of the at least onenear-to-eye display device may be adjusted according to the resolutionof the main display device, for example, the resolution of the at leastone near-to-eye display device is adjusted lower than the resolution ofthe main display device.

It can be understood by a person skilled in the art that, in theforegoing method of the embodiments of the present application, sequencenumbers of the steps do not mean execution order, and the executionorder of the steps should be determined by functions and internal logicof the steps, and should not constitute any limit to an implementationprocess of the embodiments of the present application.

An embodiment of the present application further provides a displaycontrol apparatus, and the apparatus is used to provide immersive mediaviewing experience for a user that uses a main display device to view animage (such as a text, a pattern, a video, and a game). The apparatusmay be the main display device or a near-to-eye display device, and mayalso be a part of the main display device or the near-to-eye displaydevice, or a third-party device that is independent of the main displaydevice and the near-to-eye display device, or partly belong to the maindisplay device or the near-to-eye display device. As shown in FIG. 4, adisplay control apparatus 400 of this embodiment of the presentapplication includes: a display area determining module 410 and adisplay control module 420.

The display area determining module 410 is configured to determine afirst display area, in the FOV of a user, of an image displayed on amain display device.

The FOV of a human eye vision system is about 180 degree in a horizontaldirection, and is about 135 degree in a vertical direction. At a viewingdistance where it is comfortable to view an image, the whole FOV of theuser may be not fully filled by a display of the main display device. Asshown in FIG. 2, the image displayed on the main display device fills afirst display area S1 of the FOV of the user, and there is still a verylarge unfilled region out of the first display area S1 in the FOV of theuser. In the apparatus of this embodiment of the present application,the display area determining module 410 may determine the first displayarea S1 in many manners. For example, the display area determiningmodule 410 determines the first display area S1 by detecting relativepositions of the main display device and an eye of the user and withreference to the size of the main display device; or, by collecting animage formed on the fundus of the user, or collecting the image on themain display device by using a near-to-eye display device, the displayarea determining module 410 determines the first display area S1 byusing an image processing and analysis method.

The display control module 420 is configured to use at least onenear-to-eye display device according to the first display area todisplay augmentation information related to content of the image, so asto fill the whole FOV of the user as far as possible.

The display control module 420 uses at least one near-to-eye displaydevice to display, according to the first display area, the augmentationinformation related to the image, so as to supplement the display of themain display device, and enhance viewing experience of the user byfilling the FOV of the user as far as possible. The augmentationinformation is an object, a scene, prompt information, or otherinformation that could help the user understanding the target image, andthe like that are related to the image. By displaying the augmentationinformation, the near-to-eye display device can further provideimmersive experience for the user when the user views the imagedisplayed by the main display device.

In conclusion, the apparatus of this embodiment of the presentapplication uses at least one near-to-eye display device to supplement adisplay that is provided at a suitable viewing distance of a user by adevice but doesn't fully fill the FOV of the user, and can useaugmentation information to fill the FOV of the user as far as possible,so as to provide good immersive media viewing experience for the userand to help the user understanding the target image as well.

As shown in FIG. 5, the display control module 420 may further include:a display area determining unit 421, an augmentation informationobtaining unit 422, and a display control unit 423.

The display area determining unit 421, configured to determine a seconddisplay area of the at least one near-to-eye display device in the FOVof the user according to the first display area.

According to different degrees of viewing experience to be provided forthe user, after the first display area is determined, the display areadetermining unit 421 may determine multiple possible second displayareas. As shown in FIG. 3(a), a second display area S2 may include: atleast a part of the first display area S1 (as shown in FIG. 3(a), thesecond display area includes S2 an edge part of the first display areaS1, that is, the second display area S2 overlaps an edge of the firstdisplay area S1) and a region except the first display area S1 in theFOV of the user, which is indicated by oblique lines in FIG. 3(a).Further, as shown in FIG. 3(b), the second display area S2 may onlyinclude the region except the first display area S1 in the FOV of theuser. It should be noted that vision of a human eye has a retina fovealfeature: a human eye has different visual sensitivities within the FOV,where a foveal vision region on the retina has a high visual sensitivity(for example, the foveal vision region is sensitive to colors, texture,and the like), while a peripheral vision region is more sensitive tomovement information, that is, the foveal vision region is more suitablefor viewing the image displayed on the main display device, while theperipheral vision region is more sensitive to the augmentationinformation. Therefore, the most ideal viewing experience can beprovided for the user when the second display area S2 is the peripheralvision region of the user. Correspondingly, if the first display area S1fills neither more nor less than the whole foveal vision region, thefirst display area S1 and the second display area S2 are next to eachother when the second display area S2 is located at the peripheralvision region, as shown in FIG. 3(b); if the first display area S1 doesnot fill the whole foveal vision region, a gap may exist between thesecond display area S2 and the first display area S1, as shown in FIG.3(c).

The augmentation information obtaining unit 422 is configured to obtainthe augmentation information according to the image.

The augmentation information is an object, a scene, and promptinformation or other information that could help the user understandingthe image, and the like that are related to the image. The augmentationinformation can provide immersive experience for the user when the userviews the image. In the apparatus of this embodiment of the presentapplication, the augmentation information obtaining unit 422 can obtainthe augmentation information in many manners. For example, if the imagemay include corresponding augmentation information, the augmentationinformation obtaining unit 422 directly reads the augmentationinformation from the image, and correspondingly, the apparatus of thisembodiment of the present application may include a communicationsmodule configured to obtain the augmentation information, where thecommunications module may communicate with the main display device orcommunicate with the near-to-eye display device to obtain theaugmentation information. The augmentation information obtaining unit422 may determine the augmentation information according to content ofthe image, for example, determine the augmentation information accordingto a feature of the image, where the feature of the image includes atleast one piece of the following content: a color, brightness, texture,context, movement, a display attribute (such as a frame frequency), andso on. Determining the augmentation information according to the featureof the image may be a process of determining the augmentationinformation by using an algorithm related to the AR technique which hasbeen mature in the field. For example, a tracking algorithm, an opticalflow analysis, brightness sensing, shape sensing or the like is used todetect a change in a scene in the image, and augmentation informationdisplayed outside a display screen of the main display device isdetermined with reference to an extrapolation algorithm or the like whenthe scene changes. For another example, context of the image is listenedto and comprehended by using NLP, and colors and images are used togenerate augmentation information that creates an immersive storyenvironment. Correspondingly, to obtain the augmentation information,the augmentation information obtaining unit 422 may include or use somemodules or units of the near-to-eye display device to obtain the imagedisplayed on the main display device, so as to determine theaugmentation information. The augmentation information in theembodiments of the present application is mainly visual information.

The display control unit 423 is configured to use the at least onenear-to-eye display device to display the augmentation information inthe second display area.

After the display area and display content of the at least onenear-to-eye display device are determined, the at least one near-to-eyedisplay device can be used to display the augmentation information inthe second display area, so as to provide immersive media viewingexperience for the user and to help the user understanding the targetimage as well.

It should be noted that, in order to ensure a viewing effect, thespatial position of the augmentation information displayed by the atleast one near-to-eye display device needs to be associated with thespatial position of content displayed by the main display device. Whenthe user rotates or moves the main display device, the first displayarea changes, and the second display area displayed by the at least onenear-to-eye display device changes accordingly.

Furthermore, in one possible implementation of the present application,a combination of a real image displayed by the main display device and avirtual image displayed by the at least one near-to-eye display devicemay be presented to the target user finally. For example, the maindisplay device may be a mobile phone, tablet computer, or a desktopcomputer, the at least one near-to-eye display is a smart glasses wornby the target user. In another possible implementation of the presentapplication, a combination of a virtual image displayed by the maindisplay device and anther virtual image displayed by the at least onenear-to-eye display device may be presented to the target user finally.In this case, the main display device may be a near-to-eye displaydevice which could or could not be one of the at least one near-to-eyedisplay device or not. However, in order that a display of thenear-to-eye display device does not interfere with a display on the maindisplay device and viewed by the user, but supplements or even improvesthe viewing experience, the augmentation information displayed by the atleast one near-to-eye display device should be presented as naturally aspossible whether it's a combination of a real image and a virtual imageor of two virtual images presented finally. Therefore, in order toensure the viewing effect, the apparatus of this embodiment of thepresent application further includes:

an adjusting module 430, configured to adjust a display parameter of theat least one near-to-eye display device, so as to present theaugmentation information to the user clearly.

The display parameter of the at least one near-to-eye display device maybe adjusted according to a position relationship between the maindisplay device and the eye of the user. For example, the augmentationinformation displayed by the at least one near-to-eye display device ispresented in a same plane as the image display by the main displaydevice, and correspondingly, the display parameter of the at least onenear-to-eye display device should be adjusted according to a distancebetween the main display device and the eye of the user.

The display parameter of the at least one near-to-eye display device mayalso be adjusted according to a display feature of the main displaydevice.

The display feature may include: curvature of a display plane of themain display device. When the main display device has a curved displayscreen, the display parameter of the at least one near-to-eye displaydevice should be adjusted accordingly, so that the display of the atleast one near-to-eye display device and the display of the main displaydevice can converge naturally and smoothly.

The display feature may further include: resolution of the main displaydevice. According to the retina foveal feature of the human eye vision,the main display device may display the image with relatively highresolution, while the augmentation information displayed by thenear-to-eye display device does not necessarily need resolution as highas that of the main display device. Resolution of the at least onenear-to-eye display device may be adjusted according to the resolutionof the main display device, for example, the resolution of the at leastone near-to-eye display device is adjusted smaller than the resolutionof the main display device.

FIG. 7 is a schematic structural diagram of still another displaycontrol apparatus 700 according to an embodiment of the presentapplication. Example implementation of the display control apparatus 700is not limited by this embodiment of the present application. As shownin FIG. 7, the display control apparatus 700 may include:

a processor 710, a communications interface 720, a memory 730, and acommunications bus 740.

The processor 710, the communications interface 720, and the memory 730communicate with each other through the communications bus 740.

The communications interface 720 is configured to communicate with anetwork element such as a client.

The processor 710 is configured to execute a program 732, andspecifically may execute relevant steps in the foregoing methodembodiment.

Specifically, the program 732 may include program code, where theprogram code includes a computer operation instruction.

The processor 710 may be a central processing unit (CPU), or anapplication specific integrated circuit (ASIC), or may be configured asone or more integrated circuits that implement the embodiments of thepresent application.

The memory 730 is configured to store the program 732. The memory 730may include a high speed random access memory (RAM), also may furtherinclude a non-volatile memory, such as at least one disk memory. Theprogram 732 may specifically be configured to enable the display controlapparatus 700 to execute the following steps:

determining a first display area, in the FOV of a user, of an imagedisplayed on a main display device; and

using at least one near-to-eye display device according to the firstdisplay area to display augmentation information related to the image.

For example implementation of the steps in the program 732, referencemay be made to corresponding descriptions in corresponding steps andunits in the foregoing embodiments, and details are not described hereinagain. It can be clearly known by a person skilled in the art that, tomake the description convenient and concise, for specific workingprocesses of the devices and modules described above, reference may bemade to corresponding process descriptions in the foregoing methodembodiments, and details are not described herein again.

As shown in FIG. 8, an embodiment of the present application furtherprovides a near-to-eye display device 800, including the display controlapparatus in the embodiments shown in FIG. 4 to FIG. 7.

In the embodiment of the present aspect, the near-to-eye display devicemay be smart glasses, a helmet, goggles, and so on.

It can be realized by a person of ordinary skill in the art that, unitsand method steps described with reference to the embodiments disclosedin this specification can be implemented by electronic hardware or acombination of computer software and electronic hardware. Whether thesefunctions are actually executed in a hardware or software form dependson specific applications and design constraints of the technicalsolution. A person skilled in the art may use different methods toimplement the described function for each specific application, but suchimplementation should not be considered beyond the scope of the presentapplication.

If the function is implemented in a form of a software functional unitand is sold or used as an independent product, the product can be storedin a computer-readable storage medium. Based on this understanding, thetechnical solution of the present application essentially, or a part, ofthe technical solution, that contributes to the prior art, or a part ofthe technical solution may be embodied in a form of a software product;the computer software product is stored in a storage medium and includesa number of instructions that enable a computer device (which may be apersonal computer, a server, a network device, or the like) to executeall or some of the steps of the method in the embodiments of the presentapplication. The foregoing storage medium includes all kinds of mediumsthat can store program code, such as a USB flash drive, a mobile harddisk, a read-only memory (ROM), a RAM, a magnetic disk, or a compactdisc.

The foregoing embodiments are only used to describe the presentapplication, but not to limit the present application. A person ofordinary skill in the art can still make various alterations andmodifications without departing from the spirit and scope of the presentapplication; therefore, all equivalent technical solutions also fallwithin the scope of the present application, and the patent protectionscope of the present application should be subject to the claims.

What is claimed is:
 1. A method, comprising: determining, by a devicecomprising a processor, a first display area, in a field of view (FOV)of a user, of an image displayed on a main display device; and using atleast one near-to-eye display device according to the first display areato display augmentation information related to the image.
 2. The methodof claim 1, wherein the using the at least one near-to-eye displaydevice comprises: determining, according to the first display area, asecond display area of the at least one near-to-eye display device inthe FOV of the user; obtaining the augmentation information according tothe image; and using the at least one near-to-eye display device todisplay the augmentation information in the second display area.
 3. Themethod of claim 2, wherein the second display area comprises: at least apart of the first display area and a region except the first displayarea in the FOV of the user.
 4. The method of claim 2, wherein thesecond display area comprises: a region except the first display area inthe FOV of the user.
 5. The method of claim 4, wherein the seconddisplay area comprises: a peripheral vision region of the user.
 6. Themethod of claim 2, wherein the obtaining the augmentation informationaccording to the image comprises: obtaining the augmentation informationaccording to a feature of the image.
 7. The method of claim 6, whereinthe feature of the image comprises at least one of a color, brightness,texture, context, movement, or a display attribute.
 8. The method ofclaim 1, further comprising: adjusting a display parameter of the atleast one near-to-eye display device.
 9. The method of claim 8, whereinthe adjusting the display parameter of the at least one near-to-eyedisplay device comprises: adjusting the display parameter of the atleast one near-to-eye display device according to a positionrelationship between the main display device and an eye of the user. 10.The method of claim 8, wherein the adjusting the display parameter ofthe at least one near-to-eye display device comprises: adjusting thedisplay parameter of the at least one near-to-eye display deviceaccording to a display feature of the main display device.
 11. Themethod of claim 10, wherein the display feature comprises: a curvatureof a display screen of the main display device.
 12. The method of claim10, wherein the display feature comprises: a resolution of the maindisplay device.
 13. The method of claim 1, wherein the augmentationinformation comprises at least one of an object, a scene or promptinformation that are related to content of the image.
 14. An apparatus,comprising: a processor, coupled to a memory, that executes orfacilitates execution of executable modules, comprising: a display areadetermining module configured to determine a first display area, in afield of view (FOV) of a user, of an image displayed on a main displaydevice; and a display control module configured to use at least onenear-to-eye display device according to the first display area todisplay augmentation information related to the image.
 15. The apparatusof claim 14, wherein the display control module comprises: a displayarea determining unit configured to determine, according to the firstdisplay area, a second display area of the at least one near-to-eyedisplay device in the FOV of the user; an augmentation informationobtaining unit configured to obtain the augmentation informationaccording to the image; and a display control unit configured to use theat least one near-to-eye display device to display the augmentationinformation in the second display area.
 16. The apparatus of claim 15,wherein the augmentation information obtaining unit obtains theaugmentation information according to a feature of the image.
 17. Theapparatus of claim 14, wherein the executable modules further comprise:an adjusting module configured to adjust a display parameter of the atleast one near-to-eye display device.
 18. The apparatus of claim 17,wherein the adjusting module adjusts the display parameter of the atleast one near-to-eye display device according to a positionrelationship between the main display device and an eye of the user. 19.The apparatus of claim 17, wherein the adjusting module adjusts thedisplay parameter of the at least one near-to-eye display deviceaccording to a display feature of the main display device.
 20. Anear-to-eye display device, comprising the apparatus of claim
 14. 21. Anon-transitory computer readable medium, comprising at least oneexecutable instruction, which, in response to execution, causes a systemcomprising a processor to perform operations, comprising: determining afirst display area, in a field of view (FOV) of a user, of an imagedisplayed on a main display device; and using at least one near-to-eyedisplay device according to the first display area to displayaugmentation information related to the image.
 22. A device for displaycontrol, comprising a processor and a memory, wherein the memory storesexecutable instructions, the processor being connected with the memoryvia a communication bus, and when the device is operating, the processorexecutes or facilitates execution of the executable instructions storedby the memory to cause the device to perform operations, comprising:determining a first display area, in a field of view (FOV) of a user, ofan image displayed on a main display device; and using at least onenear-to-eye display device according to the first display area todisplay augmentation information related to the image.